Biopsy support device, endoscope device, biopsy support method, and biopsy support program

ABSTRACT

A biopsy support device that supports an inspection using a living tissue sampled by a sampling instrument which is used by being inserted into an endoscope having an imaging element, the biopsy support device including a processor configured to recognize that the living tissue has been sampled by the sampling instrument on the basis of captured image data obtained by capturing an image using the imaging element; and generate identification information corresponding to the living tissue in a case where the living tissue is recognized to have been sampled.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of PCT International Application No.PCT/JP2018/040541 filed on Oct. 31, 2018, which claims priority under 35U.S.C § 119(a) to Japanese Patent Application No. 2017-212172 filed onNov. 1, 2017. Each of the above applications is hereby expresslyincorporated by reference, in its entirety, into the presentapplication.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to a biopsy support device, an endoscopedevice, a biopsy support method, and a biopsy support program.

2. Description of the Related Art

In medical sites, a living tissue, such as a polyp, is sampled from theinside of a living body, and pathological inspection, biochemicalanalysis, genomic analysis, and the like are performed on the livingtissue.

In the inspection of a digestive organ or the like using an endoscopedevice, a plurality of living tissues may be sampled in a singleinspection. In this case, as described in JP2001-095749A,JP2006-230490A, and WO2006/038634 below, the sampled living tissues aremanaged in association with captured images in a case where the livingtissues are sampled. This is important for performing highly accuratediagnosis.

JP2001-095749A describes a system in which, in a case where it isrecognized that a living tissue has been sampled on the basis of anoperation signal of a sampling instrument for sampling the livingtissue, the number of the living tissue, a captured image at the time ofsampling, and the like are recorded as the data of the sampled livingtissue.

JP2006-230490A describes a system in which, in a case where the samplinginstrument passes through a place where a wireless tag writer isinstalled after a living tissue has been sampled by the samplinginstrument for sampling the living tissue, a captured image when theliving tissue is sampled, a time point at which the living tissue hasbeen sampled, and identification information associated with patientinformation or the like are written on the wireless tag fixed to thesampling instrument.

WO2006/038634 describes a system in which, in a case where a livingtissue is sampled using a sampling instrument for sampling the livingtissue and a sampling determination button is pressed, the living tissueis discharged into a container to which identification information isadded, and the identification information of the container is associatedwith a captured image at the time of sampling the living tissue.

SUMMARY OF THE INVENTION

As described in JP2001-095749A, in the method of recognizing that theliving tissue has been sampled on the basis of the operation signal ofthe sampling instrument, in a case where an operating member foroperating the sampling instrument is erroneously operated, there is apossibility that erroneous recognition will occur. Additionally, in thismethod, a configuration for transmitting the operation signal of thesampling instrument to the control unit is required, and cost isrequired for system construction.

As described in JP2006-230490A, in the method of recognizing that theliving tissue has been sampled by inserting and removing the samplinginstrument, for example, in a case where the use of the samplinginstrument is stopped halfway, there is a possibility that erroneousrecognition will occur.

As described in WO2006/038634, in the method of recognizing that theliving tissue has been sampled by pressing the sampling determinationbutton, in a case where the sampling determination button is erroneouslyoperated, there is a possibility that erroneous recognition will occur.Additionally, in this method, there are, for example, problems that theoperation of the sampling determination button is cumbersome, and extracost for the sampling determination button is involved.

The invention has been made in view of the above circumstances, and anobject thereof is to provide a biopsy support device, an endoscopedevice comprising the same, a biopsy support method, and a biopsysupport program capable of accurately recognizing that a living tissuehas been sampled and improving diagnostic accuracy by inspection usingthe living tissue.

The biopsy support device of the invention is a biopsy support devicethat supports an inspection using a living tissue sampled by a samplinginstrument for sampling a living tissue, which is used by being insertedinto an endoscope having an imaging element, the biopsy support devicecomprising a sampling recognition unit that recognizes that the livingtissue has been sampled by the sampling instrument on the basis ofcaptured image data obtained by capturing an image using the imagingelement; and an identification information generation unit thatgenerates identification information corresponding to a living tissue ina case where the sampling recognition unit recognizes that the livingtissue has been sampled.

An endoscope device according to the invention comprises the biopsysupport device and the endoscope.

The biopsy support method of the invention is a biopsy support method ofsupporting an inspection using a living tissue sampled by a samplinginstrument for sampling a living tissue, which is used by being insertedinto an endoscope having an imaging element, the biopsy support methodcomprising a sampling recognition step of recognizing that the livingtissue has been sampled by the sampling instrument on the basis ofcaptured image data obtained by capturing an image using the imagingelement; and an identification information generation step of generatingidentification information corresponding to a living tissue in a casewhere it is recognized that the living tissue has been sampled in thesampling recognition step.

The biopsy support program of the invention is a biopsy support programof supporting an inspection using a living tissue sampled by a samplinginstrument for sampling a living tissue, which is used by being insertedinto an endoscope having an imaging element, the biopsy support programcauses a computer to execute: a sampling recognition step of recognizingthat the living tissue has been sampled by the sampling instrument onthe basis of captured image data obtained by capturing an image usingthe imaging element; and an identification information generation methodof generating identification information corresponding to a livingtissue in a case where it is recognized that the living tissue has beensampled in the sampling recognition step.

According to the invention, it is possible to provide the biopsy supportdevice, the endoscope device comprising the biopsy support device, andthe biopsy support method and the biopsy support program capable ofaccurately recognizing that a living tissue has been sampled andimproving diagnostic accuracy by an inspection using the living tissue.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view illustrating a schematic configuration of an endoscopedevice 100 that is an embodiment of the invention.

FIG. 2 is a front view of a distal end part 10C in the endoscope device100 illustrated in FIG. 1.

FIG. 3 is a schematic view illustrating an internal configuration of theendoscope device 100 illustrated in FIG. 1.

FIG. 4 is a view illustrating functional blocks of a system control unit44 of the control device 4 illustrated in FIG. 3.

FIG. 5 is a view illustrating an example of captured image data IMobtained by imaging the inside of a living body with an imaging element23 in a state where biopsy forceps 28 protrude from a forceps hole 29.

FIG. 6 is a view illustrating an example of the captured image data IMobtained by imaging the living body with the imaging element 23 in astate where the biopsy forceps 28 protrude from the forceps hole 29.

FIG. 7 is a view illustrating an example of the captured image data IMobtained by imaging the living body with the imaging element 23 in astate where the biopsy forceps 28 protrude from the forceps hole 29.

FIG. 8 is a flowchart for explaining the operation of a system controlunit 44 illustrated in FIG. 4.

FIG. 9 is a flowchart for explaining a first modification example of theoperation of the system control unit 44 illustrated in FIG. 4.

FIG. 10 is a flowchart for explaining a second modification example ofthe operation of the system control unit 44 illustrated in FIG. 4.

FIG. 11 is a view illustrating a modification example of the functionalblocks of the system control unit 44 illustrated in FIG. 4.

FIG. 12 is a view illustrating a modification example of the functionalblocks of the system control unit 44 illustrated in FIG. 4.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, embodiments of the invention will be described withreference to the drawings.

FIG. 1 is a view illustrating a schematic configuration of an endoscopedevice 100 that is an embodiment of the invention.

As illustrated in FIG. 1, the endoscope device 100 comprises anendoscope 1, and a body part 2 including a control device 4 and a lightsource device 5 to which the endoscope 1 is connected.

A display device 7 that displays a captured image or the like, an inputunit 6 that is an interface for inputting various kinds of informationto the control device 4, and a printing device 8 that performs printingon a seal attached to a storage container to be described below areconnected to the control device 4. The control device 4 controls theendoscope 1, the light source device 5, the display device 7, and theprinting device 8.

The endoscope 1 comprises an insertion part 10 that is a tubular memberextending in one direction and is inserted into a subject, an operatingpart 11 that is provided at a proximal end part of the insertion part10, an angle knob 12 provided adjacent to the operating part 11, and auniversal cord 13 including connector parts 13A and 13B that attachablyand detachably connect the endoscope 1 to the light source device 5 andthe control device 4, respectively. The insertion part 10 is providedwith operating members for performing an observation mode switchingoperation, an imaging and recording operation a forceps operation, anair and water supply operation, a suction operation, and the like.

In addition, although omitted in FIG. 1, various channels, such as aforceps hole 29 (refer to FIG. 3) for inserting biopsy forceps 28 thatis a sampling instrument for sampling a living tissue, such as cells orpolyps, an air and water supply channel, and a suction channel, areprovided inside the operating part 11 and the insertion part 10.

The insertion part 10 is constituted of a flexible part 10A that hasflexibility, a bending part 10B provided at a distal end of the flexiblepart 10A, and a hard distal end part 10C provided at a distal end of thebending part 10B.

The bending part 10B is configured to be bendable by the rotationalmovement operation of the angle knob 12. Depending on regions of thesubject in which the endoscope 1 is used, the bending part 10B can bebent in an optional direction and at an optional angle and the distalend part 10C can be directed in a desired direction.

FIG. 2 is a front view of the distal end part 10C in the endoscopedevice 100 illustrated in FIG. 1.

The distal end surface 10D of the distal end part 10C is substantiallycircular, and the distal end surface 10D is provided with an objectivelens 21, an illumination lens 50, a forceps hole 29 for taking in andout the above-described biopsy forceps 28, and an air and water supplynozzle 60 for performing air and water supply.

FIG. 3 is a schematic view illustrating an internal configuration of theendoscope device 100 illustrated in FIG. 1.

The light source device 5 comprises a light source control unit 51 and alight source unit 52.

The light source unit 52 generates illumination light for irradiatingthe subject. The illumination light emitted from the light source unit52 enters a light guide 53 built in the universal cord 13, and isemitted to the subject through the illumination lens 50 provided at thedistal end part 10C of the insertion part 10.

A white light source that emits white light, a plurality of lightsources including the white light source and a light source (forexample, a blue light source that emits blue light) that emits othercolor light, or the like is used as the light source unit 52. Aplurality of illumination lenses 50 may be provided in conformity withthe type of light emitted from the light source unit 52 on the distalend surface 10D of the distal end part 10C.

The light source control unit 51 is connected to a system control unit44 of the control device 4. The light source control unit 51 controlsthe light source unit 52 on the basis of a command from the systemcontrol unit 44.

The distal end part 10C of the endoscope 1 is provided with the imagingoptical system including the objective lens 21 and a lens group 22, animaging element 23 that images the subject through the imaging opticalsystem, an analog/digital converter circuit (ADC) 24, a memory 25, suchas a random access memory (RAM), a communication interface (I/F) 26, animaging control unit 27, the forceps hole 29 in which the biopsy forceps28 is housed, and the light guide 53 for guiding the illumination lightemitted from the light source unit 52 to the illumination lens 50.

The biopsy forceps 28 has an openable and closable gripping part 28 afor gripping an object. The biopsy forceps 28 can grip a living tissueby two movable parts constituting the gripping part 28 a.

The light guide 53 extends from the distal end part 10C to the connectorpart 13A of the universal cord 13. The illumination light emitted fromthe light source unit 52 of the light source device 5 is allowed toenter the light guide 53 in a state where the connector part 13A of theuniversal cord 13 is connected to the light source device 5.

As the imaging element 23, a charge coupled device (CCD) image sensor ora complementary metal oxide semiconductor (CMOS) image sensor is used.

The imaging element 23 has a light-receiving surface on which aplurality of pixels are two-dimensionally arranged, converts an opticalimage formed on the light-receiving surface by the above imaging opticalsystem into an electrical signal (imaging signal) in each pixel, andoutputs the converted electrical signal to the ADC 24. As the imagingelement 23, for example, one on which color filters, such as anelementary color or a complementary color, are mounted, is used. A setof the imaging signals output from the respective pixels of thelight-receiving surface of the imaging element 23 is referred to ascaptured image signals.

In addition, in a case where one in which the spectrum of the whitelight emitted from the white light source is divided in a time-divisionmanner by a plurality of color filters to generate the illuminationlight is used as the light source unit 52, one on which no color filteris mounted may be used as the imaging element 23.

The imaging element 23 may be disposed at the distal end part 10C in astate where the light-receiving surface is perpendicular to an opticalaxis Ax of the objective lens 21, or may be disposed at the distal endpart 10C in a state where the light-receiving surface is parallel to theoptical axis Ax of the objective lens 21.

The imaging optical system provided in the endoscope 1 is constituted ofoptical members (including the above lens group 22), such as a lens anda prism, which are present on an optical path of the light from thesubject between the imaging element 23 and the objective lens 21, andthe objective lens 21. There is also a case where the imaging opticalsystem is constituted of only the objective lens 21.

The ADC 24 converts the imaging signal output from the imaging element23 into a digital signal having a predetermined number of bits.

The memory 25 temporarily records the imaging signal digitally convertedby the ADC 24.

The communication I/F 26 is connected to a communication interface (I/F)41 of the control device 4. The communication I/F 26 transmits theimaging signal recorded in the memory 25 to the control device 4 througha signal line within the universal cord 13.

The imaging control unit 27 is connected to the system control unit 44of the control device 4 via the communication I/F 26. The imagingcontrol unit 27 controls the imaging element 23, the ADC 24, and thememory 25 on the basis of a command from the system control unit 44 tobe received by the communication I/F 26.

The control device 4 comprises the communication I/F 41 connected to thecommunication I/F 26 of the endoscope 1 by the universal cord 13, asignal processing unit 42, a display controller 43, the system controlunit 44, and a recording medium 45.

The communication I/F 41 receives the imaging signal transmitted fromthe communication I/F 26 of the endoscope 1 and transmits the imagingsignal to the signal processing unit 42.

The signal processing unit 42 has a memory for temporarily recording animaging signal received from the communication I/F 41 built therein, andprocesses captured image signals that are a set of the imaging signalsrecorded in the memory such as demosaicing processing orgamma-correction processing to generate captured image data in such aformat that object recognition to be described below is allowed. Thecaptured image data generated by the signal processing unit 42 isrecorded on the recording medium 45, such as a hard disk or a flashmemory.

The display controller 43 causes the display device 7 to display acaptured image based on the captured image data generated by the signalprocessing unit 42.

The system control unit 44 controls the respective units of the controldevice 4, and sends commands to the imaging control unit 27 of theendoscope 1 and the light source control unit 51 of the light sourcedevice 5, and integrally controls the entire endoscope device 100.

The system control unit 44 performs the control of the imaging element23 via the imaging control unit 27. Additionally, the system controlunit 44 performs the control of the light source unit 52 via the lightsource control unit 51.

The system control unit 44 includes various processors that execute aprogram to perform processing, a random access memory (RAM), and a readonly memory (ROM).

The various processors include a central processing unit (CPU) that is ageneral-purpose processor that executes a program to perform variouskinds of processing, a programmable logic device (PLD), which is aprocessor capable of changing a circuit configuration after manufacture,such as a field programmable gate array (FPGA), or an exclusive electriccircuit, which is a processor having a circuit configuration exclusivelydesigned to execute specific processing, such as an application specificintegrated circuit (ASIC).

The structure of these various processors is, more specifically, anelectric circuit in which circuit elements, such as semiconductorelements, are combined together.

The system control unit 44 may be constituted of one of the variousprocessors, or may be constituted of a combination (for example, acombination of a plurality of FPGAs or a combination of a CPU and anFPGA) of two or more processors of the same type or different types.

FIG. 4 is a view illustrating functional blocks of the system controlunit 44 of the control device 4 illustrated in FIG. 3.

The processor of the system control unit 44 executes a biopsy supportprogram stored in the ROM built in the system control unit 44, therebyfunctioning as a biopsy support device comprising a sampling recognitionunit 44A, an identification information generation unit 44B, amanagement unit 44C, and an output control unit 44D.

The sampling recognition unit 44A performs the object recognitionprocessing of recognizing whether or not a specific object is includedfrom the captured image data on the basis of the captured image dataobtained by processing the imaging signals, which are obtained byimaging the inside of a living body by the imaging element 23, by thesignal processing unit 42. Based on the result of the object recognitionprocessing, it is recognized that the living tissue has been sampled bythe biopsy forceps 28.

FIG. 5 is a view illustrating an example of captured image data IMobtained by imaging the inside of the living body with the imagingelement 23 in a state where the biopsy forceps 28 protrudes from theforceps hole 29. In the example illustrated in FIG. 5, the capturedimage data IM includes image information of the biopsy forceps 28 in astate where the gripping part 28 a is open.

The sampling recognition unit 44A extracts a feature amount from thecaptured image data IM exemplified in FIG. 5, and determines whether ornot the biopsy forceps 28 is included in the captured image data IM,using the feature amount. Then, in a case where the biopsy forceps 28are determined to been included in the captured image data IM, thesampling recognition unit 44A recognizes that the living tissue has beensampled by the biopsy forceps 28.

In addition, as illustrated in FIG. 6, in a case where the biopsyforceps 28 in a state where the gripping part 28 a is closed aredetermined to be included in the captured image data IM, the samplingrecognition unit 44A may recognize that the living tissue has beensampled by the biopsy forceps 28. Since the biopsy forceps 28 grips theliving tissue by closing the gripping part 28 a, according to thismethod, the fact that the living tissue has been sampled by the biopsyforceps 28 can be recognized with higher precision.

Additionally, as illustrated in FIG. 7, in a case where the biopsyforceps 28 in a state where an object P is gripped by the gripping part28 a is determined to be included in the captured image data IM, thesampling recognition unit 44A may recognize that the living tissue hasbeen sampled by the biopsy forceps 28. Since the biopsy forceps 28 gripsthe living tissue by closing the gripping part 28 a, according to thismethod, the fact that the living tissue has been sampled by the biopsyforceps 28 can be recognized with even higher precision.

Examples of the feature amount include brightness distribution,wavelets, pixel differences, histograms of oriented gradients (HOG),speeded up robust features (Joint HOG, SURF), edge of orientationhistograms (EOH), scaled invariance feature transform (SIFT), Haar-like,which is a scalar quantity obtained as a difference value of averagebrightness of a rectangular region, Joint Haar-like, sparse features,co-occurrence probability features, or Shapelet features focused on edgerelevance in a plurality of local regions.

The sampling recognition unit 44A determines whether or not the capturedimage data includes an object to be recognized through, for example, atechnique, such as template matching, principal component analysis(PCA), neural network, linear discrimination distribution, supportvector machine, boosting, adaptive boosting (Adaboost), or RealAdaboost, using these feature amounts. The sampling recognition unit 44Amay perform object recognition by deep learning.

The identification information generation unit 44B illustrated in FIG. 4generates identification information corresponding to a living tissue ina case where the sampling recognition unit 44A recognizes that theliving tissue has been sampled.

The identification information may be any information from which anobject can be identified, and an information group including a pluralityof pieces of information lined up in accordance with predeterminedorder, such as Japanese alphabets (A, I, U, E, O, . . . ), Englishalphabets (A, B, C, D, E, . . . ), hiragana character strings (i, ro,ha, ni, ho, . . . ) that are lined up in accordance with the Iroha orderof Japanese, or numerical strings (0, 1, 2, 3, 4, . . . ) includingnumbers that increase by a predetermined number (for example, “1”), ispreferably used.

The identification information generation unit 44B sets any of aplurality of pieces of information constituting this information groupas the initial information. With this initial information as a startingpoint, the identification information generation unit 44B sequentiallychanges the information within the information group to the nextinformation and generates the changed information as the identificationinformation every time the living tissue is recognized to have beensampled by the sampling recognition unit 44A. As the initialinformation, predetermined information is used.

For example, if the information group is the above-described numericalstrings and the initial information is “0”, “1” lined up next to “0” isgenerated as the identification information in a case where it isrecognized that the living tissue has been sampled by the samplingrecognition unit 44A. Then, in a case where it is recognized that thenext living tissue has been sampled, “2” lined up next to “1” isgenerated as the identification information. In this way, whenever it isrecognized that the living tissue has been sampled, the identificationinformation is sequentially updated in accordance with the sequence ofthe numerical strings.

Alternatively, for example, if the information group is the abovealphabet and the initial information is “A”, “B” lined up next to “A” isgenerated as the identification information in a case where it isrecognized that the living tissue has been sampled by the samplingrecognition unit 44A. Then, in a case where it is recognized that thenext living tissue has been sampled, “C” lined up next to “B” isgenerated as the identification information. In this way, whenever it isrecognized that the living tissue has been sampled, the identificationinformation is sequentially updated in accordance with the sequence ofthe alphabets.

In addition, the identification information may be a combination of theinformation determined by the updating as described above and personalinformation (name, patient number, and the like) of the subject.

The management unit 44C illustrated in FIG. 4 records the identificationinformation generated by the identification information generation unit44B in association with the captured image data generated by the signalprocessing unit 42 and recorded on the recording medium 45. The capturedimage data associated with the identification information is an imagedata captured in a period including a time point at which it isrecognized that a living tissue corresponding to the identificationinformation has been sampled.

The period including the time point at which it is recognized that theliving tissue has been sampled may be a period in which it is possibleto check where and in what state the sampled living tissue is presentwithin the living body.

For example, in a case where a time point at which the samplingrecognition unit 44A recognizes that the living tissue has been sampledis defined as a time point to, this period can be a period between thetime point t0 and a first time point t1 before a first time from thetime point t0. This period can also be a period between a second timepoint t2 before a second time from the time point t0 and a third timepoint t3 after a third time from the point time point t0.

The management unit 44C may set the first time or the second time andthe third time to the time which the operator of the endoscope 1 inputsto the input unit 6, or may set the first time or the second time andthe third time to a fixed value predetermined on the system side.

The captured image data obtained in the above period may be all capturedimage data (moving image data) obtained during this period, or may bestill image data obtained by extracting a portion of all the capturedimage data. Additionally, the management unit 44C may record theidentification information in association with the captured image dataobtained at the time point t0.

In a case where the identification information is generated by theidentification information generation unit 44B, the output control unit44D causes the identification information to be output from the printingdevice 8 serving as an output device.

Specifically, in a case where the identification information isgenerated by the identification information generation unit 44B, theoutput control unit 44D outputs the identification information to theprinting device 8, and causes the printing device 8 to directly outputthe identification information or output a seal printed by coding theidentification information.

This seal is attached to a storage container for storing the livingtissue by staff involved in the inspection. Then, the living tissuecorresponding to the identification information printed on the seal (theliving tissue sampled at the time point at which the seal is output) isstored in the storage container. By the control of the output controlunit 44D and the work of the staff members, the storage containerstoring the living tissue corresponding to the identificationinformation is associated with the identification information.

In addition, the printing device 8 may print the identificationinformation on the storage container itself instead of the seal.Additionally, instead of the above-described seal, a wireless tag onwhich the identification information is written may be used. In thiscase, a writing device that writes information to the wireless tag isused as the output device instead of the printing device 8.

Additionally, the attachment of the seal or the wireless tag to thestorage container may be performed automatically instead of manually bya machine connected to the control device 4.

Alternatively, there is a place where a number of empty storagecontainers are arranged, and the system control unit 44 may recognizethat an object has been put into any of these storage containers byusing a sensor or a camera, and automatically attaches theabove-mentioned seal or wireless tag to the storage container into whichthe object has been put, thereby performing the association between theidentification information and the storage container.

Next, the operation of the endoscope device 100 configured as describedabove will be described.

FIG. 8 is a flowchart for explaining the operation of the system controlunit 44 illustrated in FIG. 4.

In a case where the capturing of the moving image by the imaging element23 is started by the operation of the operating part 11, the capturedimage signals are output from the imaging element 23, the captured imagesignals are processed, and the captured image data for one frame of themoving image is sequentially generated and recorded on the recordingmedium 45. Additionally, a live view image is displayed on the displaydevice 7 on the basis of the captured image data sequentially generatedin this way.

In a case where the imaging of the moving image is started, the samplingrecognition unit 44A sequentially acquires the captured image datagenerated by the signal processing unit 42, and analyzes the acquireddata to determine whether or the living tissue has been sampled by thebiopsy forceps 28 (Step S1).

In a case where it is determined that the living tissue has not beensampled by the biopsy forceps 28 (Step S1: NO), the processing proceedsto Step S5 to be described below.

On the other hand, in a case where the sampling recognition unit 44Adetermines that the living tissue has been sampled by the biopsy forceps28 and recognizes that the living tissue has been sampled (Step S1:YES), the identification information generation unit 44B generates theidentification information corresponding to the living tissue on whichthe sampling has been recognized (Step S2). In Step S2, for example, ina case where the initial information is “0”, “1” is generated as theidentification information.

In a case where the identification information is generated in Step S2,the output control unit 44D outputs the generated identificationinformation to the printing device 8, and a seal on which theidentification information generated in Step S2 is printed is generatedby the printing device 8 (Step S3).

Additionally, in a case where the identification information isgenerated in Step S2, the management unit 44C records the generatedidentification information in association with the captured image datagenerated in the period including the time point at which thedetermination of Step S1 is YES and it is recognized that the livingtissue has been sampled, in the captured image data recorded on therecording medium 45 (Step S4).

After Step S4, in a case where an instruction to end imaging by theimaging element 23 is performed and the inspection is ended (Step S5:YES), the system control unit 44 ends the processing. On the other hand,in a case where the inspection is continued (Step S5: NO), theprocessing returns to Step S1, and the subsequent processing isrepeated.

By the above processing, for example, assuming that the operator whooperates the endoscope 1 samples a total of three living tissue from thebody of the subject with the biopsy forceps 28, a storage containerstoring a firstly sampled living tissue is in a state where a sealprinted with identification information (“1”) corresponding to theliving tissue is attached, a storage container storing a secondlysampled living tissue is in a state where a seal printed withidentification information (“2”) corresponding to the living tissue isattached, and a storage container storing a last sampled living tissueis in a state where a seal printed with identification information (“3”)corresponding to the living tissue is attached.

Additionally, in the captured image data recorded on the recordingmedium 45 by this inspection, the identification information (“1”) isrecorded in association with the captured image data in the periodincluding the time point at which it is recognized that the livingtissue corresponding to the identification information (“1”) has beensampled, the identification information (“2”) is recorded in associationwith the captured image data in the period including the time point atwhich it is recognized that the living tissue corresponding to theidentification information (“2”) has been sampled, and theidentification information (“3”) is recorded in association with thecaptured image data in the period including the time point at which itis recognized that the living tissue corresponding to the identificationinformation (“3”) has been sampled.

Then, for example, a pathological test result of a living tissue storedin a storage container on which a seal on which identificationinformation is printed is attached is managed in association with theidentification information.

Hence, while associating the pathological test result of the livingtissue with the captured image data obtained in a predetermined periodincluding the time point at which the living tissue has been sampled, itis possible to determine the disease of the subject, and an accuratediagnosis can be performed.

The endoscope device 100 recognizes that the living tissue has beensampled by the biopsy forceps 28 on the basis of the captured imagedata. For this reason, the operator of the endoscope 1 can associate aliving tissue, captured image data obtained in a case where the livingtissue is sampled, and identification information of the living tissuewith each other without performing a special operation. For this reason,the burden on the operator at the time of inspection can be reduced.Additionally, since a dedicated mechanism for notifying the systemcontrol unit 44 that the living tissue has been sampled is not required,the cost of the endoscope device 100 or the size of the endoscope 1 canbe reduced.

Additionally, according to the endoscope device 100, the presence orabsence of sampling of a living tissue is determined on the basis of thecaptured image data. Therefore, for example, in a case where the biopsyforceps 28 is sent out from the forceps hole 29 but the living tissue isnot sampled, in a case where an operation is performed in a state wherethe biopsy forceps 28 is not sent out from the forceps hole 29, or thelike, it is possible to prevent the erroneous recognition that theliving tissue has been sampled.

For example, in a method of determining whether or not the living tissueis sampled depending on whether or not the biopsy forceps 28 areincluded in the captured image data, in a case where the time duringwhich the biopsy forceps 28 are included in the captured image data isequal to or greater than a threshold value, it is determined that theliving tissue has been sampled. In this way, it is possible to identifywhether or the biopsy forceps 28 have been sent out in an attempt tosample the living tissue, or whether or the biopsy forceps 28 has beensent out for a purpose other than the erroneous operation or sampling.For this reason, it is possible to accurately determine whether or notthe living tissue has been sampled.

In the description up to this point, as the sampled living tissue isstored in a storage container, and a seal printed with identificationinformation corresponding to the living tissue is attached to thestorage container, the identification information corresponding to theliving tissue is associated with the storage container.

As a modification example, empty storage containers with seals, on whicha plurality of pieces of identification information capable of beinggenerated by the identification information generation unit 44B isprinted, is prepared in advance. Then, the operator of the endoscope 1stores a sampled living tissue to the storage container with the seal onwhich the identification information corresponding to the sampled livingtissue is printed. By this way the identification information and thestorage container prepared in advance may be associated with each other

FIG. 9 is a flowchart for explaining a first modification example of theoperation of the system control unit 44 illustrated in FIG. 4. In thefirst modification example, the printing device 8 is not connected tothe control device 4. Additionally, at the time of inspection, it isassumed that a plurality of empty storage containers to which seals onwhich the identification information generated by the identificationinformation generation unit 44B is printed are attached are prepared inadvance. In FIG. 9, the same processing as that in FIG. 8 is denoted bythe same reference numerals, and the description thereof will beomitted.

In a case where the identification information is generated in Step S2,the output control unit 44D of the system control unit 44 outputs theidentification information generated in Step S2 to the display device 7and causes the display device to display the output identificationinformation (Step S3 a).

The operator of the endoscope 1 stores the sampled living tissue in astorage container having a seal on which the identification informationdisplayed in the process of Step S3 a is printed. Accordingly, theidentification information corresponding to the sampled living tissue isassociated with the storage container.

After Step S3 a, the processing after Step S4 is performed.

As described above, according to the first modification example, thework of attaching the seal during the inspection becomes unnecessary.Therefore, the burden at the time of the inspection can be reduced.Additionally, according to the first modification example, theidentification information corresponding to the sampled living tissuecan be checked on the display device 7 together with the live viewimage. For this reason, the living tissue can be prevented from beingstored in another storage container, and the living tissue can beaccurately managed.

In addition, in the first modification example, the storage containerstoring the living tissue and the identification informationcorresponding to the living tissue may be associated with each other byhandwriting the identification information displayed on the displaydevice 7 on the storage container in which the living tissue is storedinstead of attaching a seal to the storage container. According to thisconfiguration, a system for printing identification information on aseal is not required, and the cost for constructing a system forsupporting biopsy can be reduced.

Additionally, in Step S3 a of FIG. 9, the output control unit 44Dincluded in the system control unit 44 may cause the display device 7 todisplay one piece of captured image data associated with theidentification information in Step S4 as the still image instead ofoutputting the identification information generated in Step S2 to thedisplay device 7. In this configuration, in Step S3 a, the still imageis displayed, for example, to be overlaid on the live view imagedisplayed on the display device 7. By displaying the still image, theoperator can know that the identification information has beenassociated with the captured image data.

Additionally, in Step S3 a of FIG. 9, the output control unit 44D of thesystem control unit 44 may output the identification informationgenerated in Step S2 to the display device 7, and may cause the displaydevice 7 to display the one piece of captured image data as the stillimage. In this configuration, in Step S3 a, the still image and theidentification information are displayed to be, for example, overlaid onthe live view image displayed on the display device 7. For this reason,by displaying the still image, the operator can know that theidentification information has been associated with the captured imagedata. Additionally, the living tissue can be accurately managed by thedisplayed identification information.

FIG. 10 is a flowchart for explaining a second modification example ofthe operation of the system control unit 44 illustrated in FIG. 4. Theflowchart illustrated in FIG. 10 is the same as FIG. 8 except that StepS3 a illustrated in FIG. 9 is added between Step S3 and Step S4. In FIG.10, the same processing as that in FIG. 8 is denoted by the samereference numerals, and the description thereof will be omitted.

After Step S3, the output control unit 44D of the system control unit 44outputs the identification information generated in Step S2 to thedisplay device 7, and causes the output identification information todisplay the same (Step S3 a). After Step S3 a, the processing after StepS4 is performed.

In the second modification example, in a case where the operator of theendoscope 1 stores the sampled living tissue in an empty storagecontainer, the staff attaches the seal output from the printing device 8in Step S3 to this storage container. In this case, by comparing theidentification information printed on the seal with the identificationinformation displayed on the display device 7, the storage containerstoring the living tissue can be more accurately associated with theidentification information corresponding to the living tissue.

In addition, even in Step S3 a of FIG. 10, the output control unit 44Dof the system control unit 44 may cause the display device 7 to displayone piece of captured image data associated with the identificationinformation in Step S4 as the still image instead of outputting theidentification information generated in Step S2 to the display device 7.

Alternatively, in Step S3 a of FIG. 10, the output control unit 44D ofthe system control unit 44 may output the identification informationgenerated in Step S2 to the display device 7, and may cause the displaydevice 7 to display the above one piece of captured image data as thestill image.

As described above, according to the second modification example, themanagement of the living tissue can be more accurately performed.

FIG. 11 is a view illustrating a modification example of the functionalblocks of the system control unit 44 illustrated in FIG. 4. In FIG. 11,the same components as those in FIG. 4 are denoted by the same referencenumerals, and the description thereof will be omitted.

The processor of the system control unit 44 illustrated in FIG. 11executes the biopsy support program stored in the ROM built in thesystem control unit 44, thereby functioning as the biopsy support devicecomprising the sampling recognition unit 44A, the identificationinformation generation unit 44B, the management unit 44C, the outputcontrol unit 44D, and the lesion detection unit 44E.

The lesion detection unit 44E detects a lesion site, which is a sitesuspected of a lesion, by machine learning or the like from the capturedimage data recorded on the recording medium 45.

The management unit 44C of the system control unit 44 illustrated inFIG. 11 sets, as the above first time point, the time point at which thelesion site is first detected by the lesion detection unit 44E from thecaptured image data in which the sampling of the living tissue isrecognized by the sampling recognition unit 44A.

Alternatively, the management unit 44C of the system control unit 44illustrated in FIG. 11 may set, as the second time point, the time pointat which the lesion site is first detected by the lesion detection unit44E from the captured image data in which the sampling of the livingtissue is recognized by the sampling recognition unit 44A, and may set,as the third time point, the time point at which this lesion site is notdetected after the second time point.

Alternatively, the management unit 44C of the system control unit 44illustrated in FIG. 11 may set, as the above second point, the timepoint at which the lesion site is first detected by the lesion detectionunit 44E from the captured image data in which the sampling of theliving tissue is recognized by the sampling recognition unit 44A. Themanagement unit 44C may also set, as the third time point, the timepoint at which a sampling mark generated by sampling the living tissueat the lesion site with the biopsy forceps 28 is detected after thistime point. This sampling mark may be detected from the captured imagedata recorded on the recording medium 45 by machine learning or thelike.

In other words, the management unit 44C records the identificationinformation generated by the identification information generation unit44B at the time point at which the sampling of the living tissue isrecognized by the sampling recognition unit 44A in association with someor all of all captured image data including the same lesion site as thelesion site detected by the lesion detection unit 44E (that is, the siteincluding the sampled living tissue) at that time point.

According to the configuration of the system control unit 44 illustratedin FIG. 11, captured image data showing a site including a living tissuesampled by the biopsy forceps 28 can be extracted from the capturedimage data recorded on the recording medium 45. For this reason, highlyaccurate diagnosis can be performed by comprehensively considering theresult of the pathological inspection of the living tissue and thecaptured image of the living tissue.

In addition, the configuration illustrated in FIG. 11 can be combinedwith the modification example illustrated in FIG. 9 or FIG. 10.

FIG. 12 is a view illustrating a modification example of the functionalblocks of the system control unit 44 illustrated in FIG. 4. In FIG. 12,the same components as those in FIG. 4 are denoted by the same referencenumerals, and the description thereof will be omitted.

The processor of the system control unit 44 illustrated in FIG. 12executes the biopsy support program stored in the ROM built in thesystem control unit 44, thereby functioning as the biopsy support devicecomprising the sampling recognition unit 44A, the identificationinformation generation unit 44B, the management unit 44C, the outputcontrol unit 44D, and an initial information determination unit 44F.

The initial information determination unit 44F determines the initialinformation of the identification information set by the identificationinformation generation unit 44B on the basis of the information inputfrom the input unit 6.

For example, in a case where “1” or “A” is input as setting informationof the initial information from the input unit 6, the initialinformation determination unit 44F determines the “1” or “A” as theinitial information.

In this way, by enabling the initial information of the identificationinformation to be optionally set manually, optimal management accordingto the inspection content can be performed. For example, in a case wheretwo inspections are performed on the same subject, at the time of asecond inspection, the last identification information generated in afirst inspection is set as an initial value, so that the management ofliving tissues of the same subject becomes easy.

In addition, the configuration illustrated in FIG. 12 can be combinedwith the modification examples illustrated in FIGS. 9 to 11.

As described above, the following matters are disclosed in the presentspecification.

(1) A biopsy support device that supports an inspection using a livingtissue sampled by a sampling instrument for sampling a living tissue,which is used by being inserted into an endoscope having an imagingelement, the biopsy support device comprising: a sampling recognitionunit that recognizes that the living tissue has been sampled by thesampling instrument on the basis of captured image data obtained bycapturing an image using the imaging element; and an identificationinformation generation unit that generates identification informationcorresponding to a living tissue in a case where the samplingrecognition unit recognizes that the living tissue has been sampled.

(2) The biopsy support device according to (1), further comprising: amanagement unit that records the identification information generated bythe identification information generation unit in association with thecaptured image data obtained in a period including a time point at whichit is recognized that the living tissue corresponding to theidentification information has been sampled.

(3) The biopsy support device according to (2), wherein the managementunit sets, as the period, a period between the time point and a firsttime point before a first time from the time point.

(4) The biopsy support device according to (2), wherein the managementunit sets, as the period, a period between a second time point before asecond time from the time point and a third time point after a thirdtime from the time point.

(5) The biopsy support device according to (3), further comprising: alesion detection unit that detects a site suspected of a lesion from thecaptured image data, wherein the management unit sets a time point atwhich the site detected by the lesion detection unit at the time pointis first detected by the lesion detection unit, as the first time point.

(6) The biopsy support device according to (4), further comprising: alesion detection unit that detects a site suspected of a lesion from thecaptured image data, wherein the management unit sets a time point atwhich the site detected by the lesion detection unit at the time pointis first detected by the lesion detection unit, as the second timepoint, and sets at a time point at which the site is not detected as thethird time point.

(7) The biopsy support device according to (4), further comprising: alesion detection unit that detects a site suspected of a lesion from thecaptured image data, wherein the management unit sets a time point atwhich the site detected by the lesion detection unit at the time pointis first detected by the lesion detection unit, as the second timepoint, and sets a time point at which a sampling mark generated bysampling the living tissue at the site by the sampling instrument isdetected, as the third time point.

(8) The biopsy support device according to any one of claims (1) to (7),further comprising: an output control unit that causes one or both ofthe identification information and the captured image data associatedwith the identification information to be output from an output devicein a case where the identification information is generated by theidentification information generation unit.

(9) The biopsy support device according to (8), wherein the outputdevice is a display device on which a captured image based on thecaptured image data.

(10) The biopsy support device according to (8), wherein the outputdevice includes a printing device that prints the identificationinformation or a writing device that writes the identificationinformation to a wireless tag, and wherein the output control unitoutputs the identification information from the output device.

(11) The biopsy support device according to (8), wherein the outputdevice is a display device on which a captured image based on thecaptured image data is displayed, and a printing device that prints theidentification information or a writing device that writes theidentification information to a wireless tag, and wherein the outputcontrol unit outputs the identification information from at least one ofthe display device or the writing device, and causes the captured imagedata associated with the identification information to be output fromthe display device.

(12) The biopsy support device according to any one of claims (1) to(11), wherein the sampling recognition unit recognizes that a livingtissue has been sampled in a case where the sampling instrument isincluded in the captured image data.

(13) The biopsy support device according to any one of claims (1) to(11), wherein the sampling instrument has a gripping part for grippingan object, and wherein the sampling recognition unit recognizes that aliving tissue has been sampled in a case where the sampling instrumentin a state where the gripping part is closed is included in the capturedimage data.

(14) The biopsy support device according to any one of claims (1) to(11), wherein the sampling instrument has a gripping part for grippingan object, and wherein the sampling recognition unit recognizes that aliving tissue has been sampled in a case where the sampling instrumentin a state where the object is gripped by the gripping part is includedin the captured image data.

(15) The biopsy support device according to any one of claims (1) to(14), wherein the identification information is any of the plurality ofpieces of information lined up in a predetermined order, wherein theidentification information generation unit sets any of a plurality ofpieces of information as initial information, sequentially changes theinformation to next information with the initial information as astarting point whenever it is recognized that the living tissue has beensampled by the sampling recognition unit, and generates the changedinformation as the identification information, and wherein the biopsysupport device further comprises a starting point informationdetermining unit that determines the initial information on the basis ofinformation input from an input unit for inputting information.

(16) The biopsy support device according to (15), wherein theidentification information is a number or an alphabet.

(17) An endoscope device comprising: a biopsy support device accordingto any one of (1) to (16); and the endoscope.

(18) A biopsy support method of supporting an inspection using a livingtissue sampled by a sampling instrument for sampling a living tissue,which is used by being inserted into an endoscope having an imagingelement, the biopsy support method comprising: a sampling recognitionstep of recognizing that the living tissue has been sampled by thesampling instrument on the basis of captured image data obtained bycapturing an image using the imaging element; and an identificationinformation generation step of generating identification informationcorresponding to a living tissue in a case where it is recognized thatthe living tissue has been sampled in the sampling recognition step.

(19) A biopsy support program of supporting an inspection using a livingtissue sampled by a sampling instrument for sampling a living tissue,which is used by being inserted into an endoscope having an imagingelement, the biopsy support program causes a computer to execute: asampling recognition step of recognizing that the living tissue has beensampled by the sampling instrument on the basis of captured image dataobtained by capturing an image using the imaging element; and anidentification information generation step of generating identificationinformation corresponding to a living tissue in a case where it isrecognized that the living tissue has been sampled in the samplingrecognition step.

According to the invention, it is possible to provide the biopsy supportdevice, the endoscope device comprising the biopsy support device, andthe biopsy support method and the biopsy support program capable ofaccurately recognizing that a living tissue has been sampled andimproving diagnostic accuracy by an inspection using the living tissue.

EXPLANATION OF REFERENCES

-   -   100: endoscope device    -   1: endoscope    -   2: body part    -   10: insertion part    -   10A: flexible part    -   10B: bending part    -   10C: distal end part    -   10D: distal end surface    -   11: operating Part    -   12: angle knob    -   13: universal cord    -   13A, 13B: connector part    -   6: input unit    -   7: display device    -   8: printing device    -   21: objective lens    -   Ax: optical axis    -   22: lens group    -   23: imaging element    -   24: ADC    -   25: memory    -   26: communication interface    -   27: imaging control unit    -   28: biopsy forceps    -   28 a: gripping part    -   29: forceps hole    -   4: control device    -   41: communication interface    -   42: signal processing unit    -   43 display controller    -   44: system control unit    -   44A: sampling recognition unit    -   44B: identification information generation unit    -   44C: management unit    -   44D: output control unit    -   44E: lesion detection unit    -   44F: initial information determination unit    -   45: recording medium    -   5: light source device    -   50: illumination lens    -   51: light source control unit    -   52: light source unit    -   53: light guide    -   60: air and water supply nozzle    -   IM: captured image data    -   P: object

What is claimed is:
 1. A biopsy support device that supports aninspection using a living tissue sampled by a sampling instrument whichis used by being inserted into an endoscope having an imaging element,the biopsy support device comprising: a processor configured torecognize that the living tissue has been sampled by the samplinginstrument on the basis of captured image data obtained by capturing animage using the imaging element; and generate identification informationcorresponding to the living tissue in a case where the living tissue isrecognized to have been sampled.
 2. The biopsy support device accordingto claim 1, wherein the processor further configured to record theidentification information in association with the captured image dataobtained in a period including a time point at which the living tissuecorresponding to the identification information is recognized to havebeen sampled.
 3. The biopsy support device according to claim 2, whereinthe management unit sets, as the period, a period between the time pointand a first time point before a first time from the time point.
 4. Thebiopsy support device according to claim 2, wherein the management unitsets, as the period, a period between a second time point before asecond time from the time point and a third time point after a thirdtime from the time point.
 5. The biopsy support device according toclaim 3, wherein the processor further configured to detect a sitesuspected of a lesion from the captured image data, and set a time pointat which the site detected at the time point is first detected, as thefirst time point.
 6. The biopsy support device according to claim 4wherein the processor further configured to detect a site suspected of alesion from the captured image data, set a time point at which the siteat the time point is first detected, as the second time point, and setsat a time point at which the site is not detected as the third timepoint.
 7. The biopsy support device according to claim 4, wherein theprocessor further configured to a site suspected of a lesion from thecaptured image data, sets a time point at which the site at the timepoint is first detected, as the second time point, and sets a time pointat which a sampling mark generated by sampling the living tissue at thesite by the sampling instrument is detected, as the third time point. 8.The biopsy support device according to any one of claim 1, wherein theprocessor further configured to causes one or both of the identificationinformation and the captured image data associated with theidentification information to be output from an output device in a casewhere the identification information is generated.
 9. The biopsy supportdevice according to claim 8, wherein the output device is a displaydevice on which a captured image based on the captured image data isdisplayed.
 10. The biopsy support device according to claim 2, whereinthe output device is a display device on which a captured image based onthe captured image data is displayed.
 11. The biopsy support deviceaccording to claim 8, wherein the output device includes a printingdevice that prints the identification information or a writing devicethat writes the identification information to a wireless tag, andwherein the processor configured to output the identificationinformation from the output device.
 12. The biopsy support deviceaccording to claim 8, wherein the output device is a display device onwhich a captured image based on the captured image data is displayed,and a printing device that prints the identification information or awriting device that writes the identification information to a wirelesstag, and wherein the output control unit outputs the identificationinformation from at least one of the display device or the writingdevice, and causes the captured image data associated with theidentification information to be output from the display device.
 13. Thebiopsy support device according to claim 1, wherein the processorconfigured to recognize that a living tissue has been sampled in a casewhere the sampling instrument is included in the captured image data.14. The biopsy support device according to claim 1, wherein the samplinginstrument has a gripping part for gripping an object, and wherein theprocessor configured to recognize that a living tissue has been sampledin a case where the sampling instrument in a state where the grippingpart is closed is included in the captured image data.
 15. The biopsysupport device according to claim 1, wherein the sampling instrument hasa gripping part for gripping an object, and wherein the processorconfigured to recognize that a living tissue has been sampled in a casewhere the sampling instrument in a state where the object is gripped bythe gripping part is included in the captured image data.
 16. The biopsysupport device according to claim 1, wherein the identificationinformation is any of the plurality of pieces of information lined up ina predetermined order, wherein the processor configured to set any of aplurality of pieces of information as initial information, sequentiallychanges the information to next information with the initial informationas a starting point every time the living tissue is recognized to havebeen sampled, and generate the changed information as the identificationinformation, and determine the initial information on the basis ofinformation input from an input unit for inputting information.
 17. Thebiopsy support device according to claim 16, wherein the identificationinformation is a number or an alphabet.
 18. An endoscope devicecomprising: the biopsy support device according to claim 1; and theendoscope.
 19. A biopsy support method of supporting an inspection usinga living tissue sampled by a sampling instrument which is used by beinginserted into an endoscope having an imaging element, the biopsy supportmethod comprising: recognizing that the living tissue has been sampledby the sampling instrument on the basis of captured image data obtainedby capturing an image using the imaging element; and generatingidentification information corresponding to the living tissue in a casewhere the living tissue is recognized to have been sampled.
 20. Anon-transitory computer readable medium for storing a biopsy supportprogram that supports an inspection using a living tissue sampled by asampling instrument which is used by being inserted into an endoscopehaving an imaging element, the biopsy support program causes a computerto execute a process comprising: recognizing that the living tissue hasbeen sampled by the sampling instrument on the basis of captured imagedata obtained by capturing an image using the imaging element; andgenerating identification information corresponding to the living tissuein a case where the living tissue is recognized to has been sampled.